Mutual information and conditional mean prediction error

This version: arXiv:1407.7165v1. Available from arXiv.org via the link in this recordMutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for physical models of such mechanisms and to estimate reliably from data. Furthermore, its relationship to better known statistical procedures is still poorly understood. Here we explore new connections between mutual information and regression-based dependence measures, $\nu^{-1}$, that utilise the determinant of the second-moment matrix of the conditional mean prediction error. We examine convergence properties as $\nu\rightarrow0$ and establish sharp lower bounds on mutual information and capacity of the form $\mathrm{log}(\nu^{-1/2})$. The bounds are tighter than lower bounds based on the Pearson correlation and ones derived using average mean square-error rate distortion arguments. Furthermore, their estimation is feasible using techniques from nonparametric regression. As an illustration we provide bootstrap confidence intervals for the lower bounds which, through use of a composite estimator, substantially improve upon inference about mutual information based on $k$-nearest neighbour estimators alone

Mutual information and conditional mean prediction error

This version: arXiv:1407.7165v1. Available from arXiv.org via the link in this recordMutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for physical models of such mechanisms and to estimate reliably from data. Furthermore, its relationship to better known statistical procedures is still poorly understood. Here we explore new connections between mutual information and regression-based dependence measures, $\nu^{-1}$, that utilise the determinant of the second-moment matrix of the conditional mean prediction error. We examine convergence properties as $\nu\rightarrow0$ and establish sharp lower bounds on mutual information and capacity of the form $\mathrm{log}(\nu^{-1/2})$. The bounds are tighter than lower bounds based on the Pearson correlation and ones derived using average mean square-error rate distortion arguments. Furthermore, their estimation is feasible using techniques from nonparametric regression. As an illustration we provide bootstrap confidence intervals for the lower bounds which, through use of a composite estimator, substantially improve upon inference about mutual information based on $k$-nearest neighbour estimators alone

The magnitude and colour of noise in genetic negative feedback systems

This is the final version of the article. Available from OUP via the DOI in this record.The comparative ability of transcriptional and small RNA-mediated negative feedback to control fluctuations or 'noise' in gene expression remains unexplored. Both autoregulatory mechanisms usually suppress the average (mean) of the protein level and its variability across cells. The variance of the number of proteins per molecule of mean expression is also typically reduced compared with the unregulated system, but is almost never below the value of one. This relative variance often substantially exceeds a recently obtained, theoretical lower limit for biochemical feedback systems. Adding the transcriptional or small RNA-mediated control has different effects. Transcriptional autorepression robustly reduces both the relative variance and persistence (lifetime) of fluctuations. Both benefits combine to reduce noise in downstream gene expression. Autorepression via small RNA can achieve more extreme noise reduction and typically has less effect on the mean expression level. However, it is often more costly to implement and is more sensitive to rate parameters. Theoretical lower limits on the relative variance are known to decrease slowly as a measure of the cost per molecule of mean expression increases. However, the proportional increase in cost to achieve substantial noise suppression can be different away from the optimal frontier-for transcriptional autorepression, it is frequently negligible.Funding for open access charge: MRC-EPSRC funded Fellowship in Bioinformatics (to C.G.B.)

The fidelity of dynamic signaling by noisy biomolecular networks

This is the final version of the article. Available from Public Library of Science via the DOI in this record.Cells live in changing, dynamic environments. To understand cellular decision-making, we must therefore understand how fluctuating inputs are processed by noisy biomolecular networks. Here we present a general methodology for analyzing the fidelity with which different statistics of a fluctuating input are represented, or encoded, in the output of a signaling system over time. We identify two orthogonal sources of error that corrupt perfect representation of the signal: dynamical error, which occurs when the network responds on average to other features of the input trajectory as well as to the signal of interest, and mechanistic error, which occurs because biochemical reactions comprising the signaling mechanism are stochastic. Trade-offs between these two errors can determine the system's fidelity. By developing mathematical approaches to derive dynamics conditional on input trajectories we can show, for example, that increased biochemical noise (mechanistic error) can improve fidelity and that both negative and positive feedback degrade fidelity, for standard models of genetic autoregulation. For a group of cells, the fidelity of the collective output exceeds that of an individual cell and negative feedback then typically becomes beneficial. We can also predict the dynamic signal for which a given system has highest fidelity and, conversely, how to modify the network design to maximize fidelity for a given dynamic signal. Our approach is general, has applications to both systems and synthetic biology, and will help underpin studies of cellular behavior in natural, dynamic environments.We acknowledge support from a Medical Research Council and Engineering and Physical Sciences Council funded Fellowship in Biomedical Informatics (CGB) and a Scottish Universities Life Sciences Alliance chair in Systems Biology (PSS). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript

Stochastic Simulation of Biomolecular Networks in Dynamic Environments

This is the final version of the article. Available from Public Library of Science via the DOI in this record.Simulation of biomolecular networks is now indispensable for studying biological systems, from small reaction networks to large ensembles of cells. Here we present a novel approach for stochastic simulation of networks embedded in the dynamic environment of the cell and its surroundings. We thus sample trajectories of the stochastic process described by the chemical master equation with time-varying propensities. A comparative analysis shows that existing approaches can either fail dramatically, or else can impose impractical computational burdens due to numerical integration of reaction propensities, especially when cell ensembles are studied. Here we introduce the Extrande method which, given a simulated time course of dynamic network inputs, provides a conditionally exact and several orders-of-magnitude faster simulation solution. The new approach makes it feasible to demonstrate-using decision-making by a large population of quorum sensing bacteria-that robustness to fluctuations from upstream signaling places strong constraints on the design of networks determining cell fate. Our approach has the potential to significantly advance both understanding of molecular systems biology and design of synthetic circuits.MV acknowledges support under an MRC Biomedical Informatics Fellowship. PT acknowledges support by the Royal Commission for the Exhibition of 1851. RG acknowledges support from the Leverhulme Trust (RPG-2013-171). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript

Structural change and foreign direct investment : globalization and regional economic integration

This paper investigates flows of inward and outward foreign direct investment (FDI) and FDI-to-GDP ratios in a sample of 62 countries over a 30 year time span. Using several endogenous structural break procedures (allowing for one and two break points), we find that: (1) the great majority of the series have structural breaks in the last 15 years, (2) post-break FDI and FDI/GDP ratios are substantially higher than the pre-break values, and (3) most breaks seem to be related to globalization, regional economic integration, economic growth, or political instability. Static and dynamic panel-data analy- ses accounting for and/or addressing endogeneity, simultaneity, nonstationar- ity, heterogeneity and cross-sectional dependence show that FDI is negatively related to exchange rate volatility and GDP per capita, but positively related to some regional integration agreements, trade openness, GDP, and GDP growth. Most notably, the European Union is the only regional economic integration unit found to consistently have significant and positive effects on FDI.info:eu-repo/semantics/publishedVersio

Multiplexing information flow through dynamic signalling systems

We consider how a signalling system can act as an information hub by multiplexing information arising from multiple signals. We formally define multiplexing, mathematically characterise which systems can multiplex and how well they can do it. While the results of this paper are theoretical, to motivate the idea of multiplexing, we provide experimental evidence that tentatively suggests that the NF-κB transcription factor can multiplex information about changes in multiple signals. We believe that our theoretical results may resolve the apparent paradox of how a system like NF-κB that regulates cell fate and inflammatory signalling in response to diverse stimuli can appear to have the low information carrying capacity suggested by recent studies on scalar signals. In carrying out our study, we introduce new methods for the analysis of large, nonlinear stochastic dynamic models, and develop computational algorithms that facilitate the calculation of fundamental constructs of information theory such as Kullback–Leibler divergences and sensitivity matrices, and link these methods to a new theory about multiplexing information. We show that many current models such as those of the NF-κB system cannot multiplex effectively and provide models that overcome this limitation using post-transcriptional modifications

A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation.

As modeling becomes a more widespread practice in the life sciences and biomedical sciences, researchers need reliable tools to calibrate models against ever more complex and detailed data. Here we present an approximate Bayesian computation (ABC) framework and software environment, ABC-SysBio, which is a Python package that runs on Linux and Mac OS X systems and that enables parameter estimation and model selection in the Bayesian formalism by using sequential Monte Carlo (SMC) approaches. We outline the underlying rationale, discuss the computational and practical issues and provide detailed guidance as to how the important tasks of parameter inference and model selection can be performed in practice. Unlike other available packages, ABC-SysBio is highly suited for investigating, in particular, the challenging problem of fitting stochastic models to data. In order to demonstrate the use of ABC-SysBio, in this protocol we postulate the existence of an imaginary reaction network composed of seven interrelated biological reactions (involving a specific mRNA, the protein it encodes and a post-translationally modified version of the protein), a network that is defined by two files containing 'observed' data that we provide as supplementary information. In the first part of the PROCEDURE, ABC-SysBio is used to infer the parameters of this system, whereas in the second part we use ABC-SysBio's relevant functionality to discriminate between two different reaction network models, one of them being the 'true' one. Although computationally expensive, the additional insights gained in the Bayesian formalism more than make up for this cost, especially in complex problems

A geometric analysis of fast-slow models for stochastic gene expression

Stochastic models for gene expression frequently exhibit dynamics on several different scales. One potential time-scale separation is caused by significant differences in the lifetimes of mRNA and protein; the ratio of the two degradation rates gives a natural small parameter in the resulting chemical master equation, allowing for the application of perturbation techniques. Here, we develop a framework for the analysis of a family of ‘fast-slow’ models for gene expression that is based on geometric singular perturbation theory. We illustrate our approach by giving a complete characterisation of a standard two-stage model which assumes transcription, translation, and degradation to be first-order reactions. In particular, we present a systematic expansion procedure for the probability-generating function that can in principle be taken to any order in the perturbation parameter, allowing for an approximation of the corresponding propagator probabilities to that same order. For illustrative purposes, we perform this expansion explicitly to first order, both on the fast and the slow time-scales; then, we combine the resulting asymptotics into a composite fast-slow expansion that is uniformly valid in time. In the process, we extend, and prove rigorously, results previously obtained by Shahrezaei and Swain (Proc Natl Acad Sci USA 105(45):17256–17261, 2008) and Bokes et al. (J Math Biol 64(5):829–854, 2012; J Math Biol 65(3):493–520, 2012). We verify our asymptotics by numerical simulation, and we explore its practical applicability and the effects of a variation in the system parameters and the time-scale separation. Focussing on biologically relevant parameter regimes that induce translational bursting, as well as those in which mRNA is frequently transcribed, we find that the first-order correction can significantly improve the steady-state probability distribution. Similarly, in the time-dependent scenario, inclusion of the first-order fast asymptotics results in a uniform approximation for the propagator probabilities that is superior to the slow dynamics alone. Finally, we discuss the generalisation of our geometric framework to models for regulated gene expression that involve additional stages